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DETAILED ACTION 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
1 1/27/06 has been entered. 

Response to Arguments 

Applicant's arguments filed 12/21/06 have been fully considered but they are not 
persuasive. Foremost, it should be noted that Rangan (6198833) explicitly does teach 
linked video files separate from video content by explaining output video stream 
comprises the original stream plus the synchronous data stream that contains the 
tracking data and is illustrated as leaving module (Col 6, lines 48-51). Additionally 
Rangan teaches during the tracking process, a separate data stream is created, 
synchronous to the original video stream, that contains the coordinate data for the 
center position of tracking element determined in each frame (Col 10, lines 53-56) 
Therefore, as taught by Rangan in both of these recitations, files consist of two separate 
entities: the original video stream data and the tracking data and consequently the 
video content is in fact separate from linked video files. 

Furthermore, applicant has asserted that Feinlieb fails to teach linked video files 
separate from and not embedded in video content. Examiner respectfully disagrees. It . 
should be noted that Feinlieb explicitly teaches both video image data as well as non- 
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image information, which is data contained within VBI (vertical blanking interval). It 
should be noted that these lines coincide with vertical refresh period is which the 
cathode ray tube of the television is moved form lower right hand corner of the screen 
upon completion of one frame to upper left hand corner of the screen for 
commencement of the next frame and furthermore Feinlieb teaches non-image 
information such as closed captioning data may be stored in the VBI. (Col 3 lines 49-65) 
Additionally it should be noted that applicant teaches two types of content, primary 
content (video continuous image data) such as broadcast television, transmitted cable 
shows, etc... and enhancement content which is used to enhance primary content such 
as audio, video, text, hypermedia, etc... (Col 5 lines 25-39) 

Enhancing content as taught by Feinlieb may reside on a storage medium at a 
viewers home, such as on a computer disk or CD-Rom which then can be accessed 
during playing of primary content. Feinleib teaches linked videos files for enhancing 
content separate from and not embedded in video content by teaching enhancing 
content may reside in a viewers home and is synchronized by a closed caption script of 
the primary content with the synchronization independent of how and when the 
enchancing content or primary content is delivered to the viewer computing units. (Col 6 
lines 23-30) Again, Feinleib explicitly teaches enhancing content can be delivered 
independently of the primary content and synchronized at the viewer-computing unit 
using the closed captioning script, which accompanies the primary content. (Col 9 lines 
30-40). Thus, Feinleib teaches a closed captioning script used as a timing mechanism 
for the enhancing content, for example the author might choose the phrase "oh hi how 
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are you" as the trigger for displaying a hyperlink (enhancement data). (Col 7 lines 20- 
29) Consequently Feinleib explicitly teaches enhancement data separate and not 
embedded in primary content. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rangan 
(6198833) in view of Feinleib. (6637032) 

In claim 1 , Rangan teaches an image processing system for processing video 
content in a sequence of video frames and linking one or more pixel objects embedded 
in said video content to selected data objects in a sequence of video frames by 
explaining a system is provided for tracking a moving entity in a video presentation, the 
system comprising a computer station presenting the video presentation on a display as 
a series of bitmapped frames; and a tracking module receiving the video data stream. 
(Col 3, lines 26-29); said image processing system comprising a video capture system 
for capturing a frame of said sequence of video frames to be viewed defining a captured 
video frame by showing a recording function for accepting the positions wherein the 
pixel signature (defined in the art as a local neighborhood around given pixel) most 
closely matches the image signature as the true positions of the image entity in the next 
frames. (Col 3, lines 43-46) and in FIG. 1 input data stream 15 to tracking module 13 is 
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a stream of successive bitmapped frames in a normalized resolution, required by the 
tracking module. (Col 5, lines 35-37) The authoring station can be based on virtually 
any sort of computer platform and operating system, and in a preferred embodiment, a 
PC station running MS Windows is used, in which case the input stream 16, regardless 
of protocol, is converted to a digital video format that can be interpreted and played 
back as a sequence of bitmapped frames. (Col 5, lines 37-43) Furthermore Rangan 
teaches a user interface for enabling a user to select one or more pixel objects in said 
captured frame defining selected pixel objects. (Col 4 lines 1 1-35). Additionally Rangan 
teaches a pixel object tracking system, which includes a processor, which automatically 
tracks, said selected pixel objects in other frames. (Col 3, lines 26-50). It should be 
noted that it is well known in the art that a computer system would inherently contain a 
processor. Rangan also teaches said video linking system generating one or more 
linked video files, separate from said video content (Col 6 lines 48-51, Col 10 lines 53- 
66) by explaining when tracking element 29 (Fig. 2) is positioned and activated over an 
image entity to be tracked, a signature table is created and stored (Col 8, lines 40-42) 
and upon tracking element 29 being activated the tracking module creates a table or list 
comprising pixel values associated with a target number and spatial arrangement of 
pixels associated with tracking element 29. (Col 7, lines 40-43). Although Rangan does 
not explicitly state the generation of video files, it is inherent to the invention that a table 
or list, which is created by the tracking module and subsequently stored, must implicitly 
require files for storage function. Lastly, Rangan teaches through additional editing 
processes, a moving region associated with the image entity in a display may be made 
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to be interactive and identifiable to an end user. (Col 6, lines 55-57). Rangan further 
teaches user interaction with such an image entity during viewing of a video can be 
programmed to provide additional network-stored information about that a entity to 
suitable customer premises equipment (CPE) adapted to receive and display that 
information (Col 6, lines 57-62) and such further information may be displayed, for 
example, as an overlay on the display of the dynamic video containing the subject 
image entity. (Col 6, lines 62-64) It should be noted Rangan further teaches providing 
one or more links to predetermined data objects for each pixel object. (Col 7 lines 25- 
52, Fig. 2) Nonetheless, Rangan fails to explicitly teach said video linking system 
generating one or more linked video files separate from said video content, being 
configured to identify the pixel objects by frame number and location within the frame. It 
would have been obvious to one of ordinary skill in the art at the present time the 
invention was made to utilize user editing processes and programmable capabilities of 
stored information about an image entity, as taught by Reagan, to identify the pixel 
objects by frame number and location within a frame because it is well known in the art 
that stored information about an image entity will include information about the image 
object's frame number and location within the frame in order to properly retrieve and 
display that information. Furthermore, these user programmable abilities allow 
advertisers, product promoters, or the like to present information to end users based on 
user interaction with an associate entity in a dynamic video display. (Col 6, lines 64-67) 
Reagan also teaches linked video files are synchronized with said video content. (Col 6, 
lines 48-51 and Col 10, lines 53-56) Furthermore Reagan teaches wherein said linked 
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video files are configured so that selected locations in said video frames by a pointing 
device during playback of the video content can be linked with said data objects when 
said selected locations correspond said pixel objects. (Col 7 lines 35-52) It should be 
noted that the point device as taught by Reagan is a mouse. However, Reagan does 
not explicitly teach information not embedded in video content. This is what Feinleib 
teaches. (Col 3 lines 51-65, Col 9 lines 27-39, Col 11 lines 17-27) It should be noted 
that Feinleib teaches linked videos files for enhancing content separate from and not 
embedded in video content by teaching enhancing content may reside in a viewers 
home and is synchronized by a closed caption script of the primary content with the 
synchronization independent of how and when the enchancing content or primary 
content is delivered to the viewer computing units. (Col 6 lines 23-30) Again, Feinleib 
explicitly teaches enhancing content can be delivered independently of the primary 
content and synchronized at the viewer-computing unit using the closed captioning 
script, which accompanies the primary content. (Col 9 lines 30-40). It would have been 
obvious to one of ordinary skill in the art at the present time the invention was made to 
combine the teachings of generating one or more linked video files separate from and 
not embedded in video content into the system of Reagan because enhancements to 
primary content can be timely introduced at desired junctures of the primary content. 
(Col 2 lines 14-20) 

Regarding claim 11, Rangan teaches wherein said video playback application is 
configured to determine if selected locations by a pointing device during play back of the 
video content correspond to said predetermined pixel object and provide a link to a data 
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object when said selected location corresponds to said predetermined pixel object 
which is similarly recited in the amended independent claim. (Col 7 lines 35-52) 
Claims 2-3 are rejected under 35 U.S.C. 103(a) as being unpatentable over Rangan 
(61 98833) in view of Feinleib (6637032) in further view of Vidovic (3878557). 

Consider claim 2, Rangan teaches a predetermined playback rate by showing in 
one preferred embodiment the subject video is displayed typically at 30 frames per 
second with a resolution of 352 by 240 pixels. (Col 5, lines 43-46) However, neither 
Rangan nor Feinleiv explicitly teaches said video linking system samples said video 
content at a sample rate of less than said predetermined playback rate. This is what 
Vidovic teaches. Vidovic teaches a videotape recording apparatus, which shows color 
frame pulses separated by 66 milliseconds and have a 15Hz rate (Col 23, lines 56-57 
and Fig 17B) It would have been obvious to one of ordinary skill in the art at the 
present time the invention was made to combine a videotape recording apparatus 
sampled at 15Hz as taught by Vidovic with a video linking system displaying video at 
30 frames per second as taught by Rangan in order to show the two possible phases 
of the color frame reference pulses derived from the input color video signal (Col 23, 
lines 57-60 and Fig 17) and thus, to direct in choosing the correct phase. (Col 7, lines 
54-55) 

Consider claim 3, Vidovic does not explicitly define a sample rate of three frames 
per second. However, it would have been obvious to one of ordinary skill in the art at 
the present time the invention was made to lower a 15 Hz sampling rate for videotape 
recording as taught by Vidovic to 3 Hz sampling rate because it is well known in the art 



Application/Control Number: 10/786,777 Page 9 

Art Unit: 2628 

that it is always practical to lower a sampling rate due to bandwidth or file size 
limitations. 

Claims 4-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Rangan (6198833) in view of Feinleib (6637032) in further view of Toklu (6549643). 

Regarding claim 4, the teachings of Rangan and Feinleib are given in the 
previous paragraphs of this Office Action. However, neither Rangan nor Feinleib 
explicitly teaches said video linking system is configured to identify segment breaks in 
said video content. This is what Toklu teaches. Toklu teaches video summarization 
methods typically include segmenting a video into an appropriate set of segments such 
as video "shots" and selecting one or more key-frames from the shots. (Col 1 , lines 34- 
37) It should be noted that a key-frame is defined in the art to be a frame used to 
indicate the beginning or end of a change made to the signal and therefore* an implied 
segment break. It would have been obvious to one of ordinary skill in the art at the 
present time the invention was made to combine video summarization methods 
configured to identify segment breaks as taught by Toklu with the image processing 
system as taught by Rangan in order to reduce the number of images to one or more 
key-frames to represent the content of a given shot (Col 1 , lines 43-45) and thus, to 
generate a video summary. (Col 1,. line 33). 

Regarding claim 5, the teachings of Rangan and Feinleib are given in the 
previous paragraphs of this Office Action. However, neither Rangan nor Feinleib 
explicitly teaches said segment breaks are determined by determining the median 
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average pixel values for a series of frames and comparing changes in the pixel values 
relative to the median average and indicating a segment break when the change in 
pixel values represents at least a predetermined change relative to the median 
average. This is what Toklu teaches. Toklu teaches determining median average pixel 
values for a series of frames by showing computing an average of an absolute pixel- 
based intensity difference between consecutive frames in each segment, and for each 
segment, computing a cumulative sum of the average of the absolute pixel-based 
intensity differences for the corresponding frames of the segment. (Col 3, lines 61-67) 
Toklu also teaches comparing changes in pixel values relative to median average by 
explaining selecting the first frame in each motion activity segment of a given segment 
frame if the cumulative sum of the average of the absolute pixel-based intensity 
differences for the frames of the given segment does not exceed a first predefined 
threshold. (Col 4, lines 1-5) Lastly, Toklu teaches indicating a segment break when the 
change in pixel values represents at least a predetermined change relative to the 
median average by showing selecting a predefined number of key-frames in the given 
segment uniformly, if the cumulative sum of the average of the absolute pixel-based 
intensity differences for the frames of the given segment exceeds the first predefined 
threshold. (Col 4, lines 5-9) It should be noted that a key-frame is defined in the art to 
be a frame used to indicate the beginning or end of a change made to the signal and 
therefore an implied segment break. It would have been obvious to one of ordinary 
skill in the art at the present time the invention was made to combine determining the 
average pixel values for a series of frames, comparing changes in pixel values relative 
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to the average and indicating a segment break when the change in pixel values 
represents at least a predetermined change relative to the median average as taught 
by Toklu with the image processing system as taught by Rangan in order to measure a 
temporal activity curve for dissimilarity based on frame differences. (Col 3, lines 60-62) 
and thus, make possible in the system and method for selecting key-frames from video 
data. (Col 3, lines 51-59) 

Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Rangan 
(6198833) in view of Feinleib (6637032) in further view of Hunke. (5912980) 

Regarding claim 12, Hunke teaches object-tracking system automatically 
compensates for changes in the color values of a pixel object due to lighting changes. 
(Col 3 lines 20-40, Col 7 line 9-40) It should be noted that Hunke teaches ITCC is 
updated regarding colors occurring in tracked object allowing the system to 
automatically adjust to changing lighting conditions and appearance of the tracked 
target. It would have been obvious to one of ordinary skill in the art at the present time 
the invention was made to combine automatically compensating for changes in color 
values due to lighting changes because providing the functionality of rapid adjustments 
to changing lighting conditions and appearance of tracked target can be achieved, and 
thus, a improved appearance of the tracked target can be achieved. (Col 3 lines 38-40) 

Conclusion 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kevin K. Xu whose telephone number is 571-272-7747. 
The examiner can normally be reached on 8:30AM - 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman can be reached on 571-272-7653. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

\C X' Kevin Xu 
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